{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# set tf 1.x for colab\n",
    "%tensorflow_version 1.x"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Fine-tuning InceptionV3 for flowers classification\n",
    "\n",
    "In this task you will fine-tune InceptionV3 architecture for flowers classification task.\n",
    "\n",
    "InceptionV3 architecture (https://research.googleblog.com/2016/03/train-your-own-image-classifier-with.html):\n",
    "<img src=\"images/inceptionv3.png\" style=\"width:70%\">\n",
    "\n",
    "Flowers classification dataset (http://www.robots.ox.ac.uk/~vgg/data/flowers/102/index.html) consists of 102 flower categories commonly occurring in the United Kingdom. Each class contains between 40 and 258 images:\n",
    "<img src=\"images/flowers.jpg\" style=\"width:70%\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Import stuff"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "import sys\n",
    "sys.path.append(\"..\")\n",
    "import grading\n",
    "import download_utils"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# !!! remember to clear session/graph if you rebuild your graph to avoid out-of-memory errors !!!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "download_utils.link_all_keras_resources()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2017-09-03T13:00:39.210959Z",
     "start_time": "2017-09-03T13:00:39.057800Z"
    },
    "collapsed": true,
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "import tensorflow as tf\n",
    "import keras\n",
    "from keras import backend as K\n",
    "import numpy as np\n",
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "print(tf.__version__)\n",
    "print(keras.__version__)\n",
    "import cv2  # for image processing\n",
    "from sklearn.model_selection import train_test_split\n",
    "import scipy.io\n",
    "import os\n",
    "import tarfile\n",
    "import keras_utils\n",
    "from keras_utils import reset_tf_session "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Fill in your Coursera token and email\n",
    "To successfully submit your answers to our grader, please fill in your Coursera submission token and email"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "grader = grading.Grader(assignment_key=\"2v-uxpD7EeeMxQ6FWsz5LA\", \n",
    "                        all_parts=[\"wuwwC\", \"a4FK1\", \"qRsZ1\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "# token expires every 30 min\n",
    "COURSERA_TOKEN = ### YOUR TOKEN HERE\n",
    "COURSERA_EMAIL = ### YOUR EMAIL HERE"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Load dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Dataset was downloaded for you, it takes 12 min and 400mb.\n",
    "Relevant links (just in case):\n",
    "- http://www.robots.ox.ac.uk/~vgg/data/flowers/102/index.html\n",
    "- http://www.robots.ox.ac.uk/~vgg/data/flowers/102/102flowers.tgz\n",
    "- http://www.robots.ox.ac.uk/~vgg/data/flowers/102/imagelabels.mat"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "# we downloaded them for you, just link them here\n",
    "download_utils.link_week_3_resources()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Prepare images for model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2017-09-03T13:00:39.214524Z",
     "start_time": "2017-09-03T13:00:39.212453Z"
    },
    "collapsed": true,
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "# we will crop and resize input images to IMG_SIZE x IMG_SIZE\n",
    "IMG_SIZE = 250"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2017-09-10T12:46:09.790818Z",
     "start_time": "2017-09-10T12:46:09.777771Z"
    },
    "collapsed": true,
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "def decode_image_from_raw_bytes(raw_bytes):\n",
    "    img = cv2.imdecode(np.asarray(bytearray(raw_bytes), dtype=np.uint8), 1)\n",
    "    img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)\n",
    "    return img"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will take a center crop from each image like this:\n",
    "<img src=\"images/center_crop.jpg\" style=\"width:50%\">"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2017-09-03T13:00:39.393675Z",
     "start_time": "2017-09-03T13:00:39.302130Z"
    },
    "collapsed": true,
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "def image_center_crop(img):\n",
    "    \"\"\"\n",
    "    Makes a square center crop of an img, which is a [h, w, 3] numpy array.\n",
    "    Returns [min(h, w), min(h, w), 3] output with same width and height.\n",
    "    For cropping use numpy slicing.\n",
    "    \"\"\"\n",
    "    \n",
    "    cropped_img = ### YOUR CODE HERE\n",
    "    \n",
    "    # checks for errors\n",
    "    h, w, c = img.shape\n",
    "    assert cropped_img.shape == (min(h, w), min(h, w), c), \"error in image_center_crop!\"\n",
    "    \n",
    "    return cropped_img"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2017-09-03T13:00:39.506473Z",
     "start_time": "2017-09-03T13:00:39.395470Z"
    },
    "collapsed": true,
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "def prepare_raw_bytes_for_model(raw_bytes, normalize_for_model=True):\n",
    "    img = decode_image_from_raw_bytes(raw_bytes)  # decode image raw bytes to matrix\n",
    "    img = image_center_crop(img)  # take squared center crop\n",
    "    img = cv2.resize(img, (IMG_SIZE, IMG_SIZE))  # resize for our model\n",
    "    if normalize_for_model:\n",
    "        img = img.astype(\"float32\")  # prepare for normalization\n",
    "        img = keras.applications.inception_v3.preprocess_input(img)  # normalize for model\n",
    "    return img"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "# reads bytes directly from tar by filename (slow, but ok for testing, takes ~6 sec)\n",
    "def read_raw_from_tar(tar_fn, fn):\n",
    "    with tarfile.open(tar_fn) as f:\n",
    "        m = f.getmember(fn)\n",
    "        return f.extractfile(m).read()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2017-09-03T13:00:39.961301Z",
     "start_time": "2017-09-03T13:00:39.508004Z"
    },
    "collapsed": true,
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "# test cropping\n",
    "raw_bytes = read_raw_from_tar(\"102flowers.tgz\", \"jpg/image_00001.jpg\")\n",
    "\n",
    "img = decode_image_from_raw_bytes(raw_bytes)\n",
    "print(img.shape)\n",
    "plt.imshow(img)\n",
    "plt.show()\n",
    "\n",
    "img = prepare_raw_bytes_for_model(raw_bytes, normalize_for_model=False)\n",
    "print(img.shape)\n",
    "plt.imshow(img)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "## GRADED PART, DO NOT CHANGE!\n",
    "# Test image preparation for model\n",
    "prepared_img = prepare_raw_bytes_for_model(read_raw_from_tar(\"102flowers.tgz\", \"jpg/image_00001.jpg\"))\n",
    "grader.set_answer(\"qRsZ1\", list(prepared_img.shape) + [np.mean(prepared_img), np.std(prepared_img)])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "# you can make submission with answers so far to check yourself at this stage\n",
    "grader.submit(COURSERA_EMAIL, COURSERA_TOKEN)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Prepare for training"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "# read all filenames and labels for them\n",
    "\n",
    "# read filenames firectly from tar\n",
    "def get_all_filenames(tar_fn):\n",
    "    with tarfile.open(tar_fn) as f:\n",
    "        return [m.name for m in f.getmembers() if m.isfile()]\n",
    "\n",
    "all_files = sorted(get_all_filenames(\"102flowers.tgz\"))  # list all files in tar sorted by name\n",
    "all_labels = scipy.io.loadmat('imagelabels.mat')['labels'][0] - 1  # read class labels (0, 1, 2, ...)\n",
    "# all_files and all_labels are aligned now\n",
    "N_CLASSES = len(np.unique(all_labels))\n",
    "print(N_CLASSES)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2017-09-03T13:00:40.185940Z",
     "start_time": "2017-09-03T13:00:40.175758Z"
    },
    "collapsed": true,
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "# split into train/test\n",
    "tr_files, te_files, tr_labels, te_labels = \\\n",
    "    train_test_split(all_files, all_labels, test_size=0.2, random_state=42, stratify=all_labels)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "# will yield raw image bytes from tar with corresponding label\n",
    "def raw_generator_with_label_from_tar(tar_fn, files, labels):\n",
    "    label_by_fn = dict(zip(files, labels))\n",
    "    with tarfile.open(tar_fn) as f:\n",
    "        while True:\n",
    "            m = f.next()\n",
    "            if m is None:\n",
    "                break\n",
    "            if m.name in label_by_fn:\n",
    "                yield f.extractfile(m).read(), label_by_fn[m.name]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2017-09-03T13:00:40.529088Z",
     "start_time": "2017-09-03T13:00:40.423114Z"
    },
    "collapsed": true,
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "# batch generator\n",
    "BATCH_SIZE = 32\n",
    "\n",
    "def batch_generator(items, batch_size):\n",
    "    \"\"\"\n",
    "    Implement batch generator that yields items in batches of size batch_size.\n",
    "    There's no need to shuffle input items, just chop them into batches.\n",
    "    Remember about the last batch that can be smaller than batch_size!\n",
    "    Input: any iterable (list, generator, ...). You should do `for item in items: ...`\n",
    "        In case of generator you can pass through your items only once!\n",
    "    Output: In output yield each batch as a list of items.\n",
    "    \"\"\"\n",
    "    \n",
    "    ### YOUR CODE HERE"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "## GRADED PART, DO NOT CHANGE!\n",
    "# Test batch generator\n",
    "def _test_items_generator():\n",
    "    for i in range(10):\n",
    "        yield i\n",
    "\n",
    "grader.set_answer(\"a4FK1\", list(map(lambda x: len(x), batch_generator(_test_items_generator(), 3))))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "# you can make submission with answers so far to check yourself at this stage\n",
    "grader.submit(COURSERA_EMAIL, COURSERA_TOKEN)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2017-09-03T13:00:40.637615Z",
     "start_time": "2017-09-03T13:00:40.530642Z"
    },
    "collapsed": true,
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "def train_generator(files, labels):\n",
    "    while True:  # so that Keras can loop through this as long as it wants\n",
    "        for batch in batch_generator(raw_generator_with_label_from_tar(\n",
    "                \"102flowers.tgz\", files, labels), BATCH_SIZE):\n",
    "            # prepare batch images\n",
    "            batch_imgs = []\n",
    "            batch_targets = []\n",
    "            for raw, label in batch:\n",
    "                img = prepare_raw_bytes_for_model(raw)\n",
    "                batch_imgs.append(img)\n",
    "                batch_targets.append(label)\n",
    "            # stack images into 4D tensor [batch_size, img_size, img_size, 3]\n",
    "            batch_imgs = np.stack(batch_imgs, axis=0)\n",
    "            # convert targets into 2D tensor [batch_size, num_classes]\n",
    "            batch_targets = keras.utils.np_utils.to_categorical(batch_targets, N_CLASSES)\n",
    "            yield batch_imgs, batch_targets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2017-09-03T13:00:41.092659Z",
     "start_time": "2017-09-03T13:00:40.639132Z"
    },
    "collapsed": true,
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "# test training generator\n",
    "for _ in train_generator(tr_files, tr_labels):\n",
    "    print(_[0].shape, _[1].shape)\n",
    "    plt.imshow(np.clip(_[0][0] / 2. + 0.5, 0, 1))\n",
    "    break"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Training"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2017-09-10T13:16:35.109044Z",
     "start_time": "2017-09-10T13:16:35.105127Z"
    }
   },
   "source": [
    "You cannot train such a huge architecture from scratch with such a small dataset.\n",
    "\n",
    "But using fine-tuning of last layers of pre-trained network you can get a pretty good classifier very quickly."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2017-09-03T13:00:41.097588Z",
     "start_time": "2017-09-03T13:00:41.094216Z"
    },
    "collapsed": true,
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "# remember to clear session if you start building graph from scratch!\n",
    "s = reset_tf_session()\n",
    "# don't call K.set_learning_phase() !!! (otherwise will enable dropout in train/test simultaneously)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2017-09-03T13:00:41.222209Z",
     "start_time": "2017-09-03T13:00:41.098974Z"
    },
    "collapsed": true,
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "def inception(use_imagenet=True):\n",
    "    # load pre-trained model graph, don't add final layer\n",
    "    model = keras.applications.InceptionV3(include_top=False, input_shape=(IMG_SIZE, IMG_SIZE, 3),\n",
    "                                          weights='imagenet' if use_imagenet else None)\n",
    "    # add global pooling just like in InceptionV3\n",
    "    new_output = keras.layers.GlobalAveragePooling2D()(model.output)\n",
    "    # add new dense layer for our labels\n",
    "    new_output = keras.layers.Dense(N_CLASSES, activation='softmax')(new_output)\n",
    "    model = keras.engine.training.Model(model.inputs, new_output)\n",
    "    return model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2017-09-03T13:00:45.150429Z",
     "start_time": "2017-09-03T13:00:41.223777Z"
    },
    "collapsed": true,
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "model = inception()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2017-09-03T13:00:45.252883Z",
     "start_time": "2017-09-03T13:00:45.152062Z"
    },
    "collapsed": true,
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "model.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2017-09-03T13:00:45.273005Z",
     "start_time": "2017-09-03T13:00:45.254250Z"
    },
    "collapsed": true,
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "# how many layers our model has\n",
    "print(len(model.layers))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2017-09-03T13:00:45.370171Z",
     "start_time": "2017-09-03T13:00:45.274354Z"
    },
    "collapsed": true,
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "# set all layers trainable by default\n",
    "for layer in model.layers:\n",
    "    layer.trainable = True\n",
    "    if isinstance(layer, keras.layers.BatchNormalization):\n",
    "        # we do aggressive exponential smoothing of batch norm\n",
    "        # parameters to faster adjust to our new dataset\n",
    "        layer.momentum = 0.9\n",
    "    \n",
    "# fix deep layers (fine-tuning only last 50)\n",
    "for layer in model.layers[:-50]:\n",
    "    # fix all but batch norm layers, because we neeed to update moving averages for a new dataset!\n",
    "    if not isinstance(layer, keras.layers.BatchNormalization):\n",
    "        layer.trainable = False"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2017-09-03T13:00:45.494833Z",
     "start_time": "2017-09-03T13:00:45.371512Z"
    },
    "collapsed": true,
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "# compile new model\n",
    "model.compile(\n",
    "    loss='categorical_crossentropy',  # we train 102-way classification\n",
    "    optimizer=keras.optimizers.adamax(lr=1e-2),  # we can take big lr here because we fixed first layers\n",
    "    metrics=['accuracy']  # report accuracy during training\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# we will save model checkpoints to continue training in case of kernel death\n",
    "model_filename = 'flowers.{0:03d}.hdf5'\n",
    "last_finished_epoch = None\n",
    "\n",
    "#### uncomment below to continue training from model checkpoint\n",
    "#### fill `last_finished_epoch` with your latest finished epoch\n",
    "# from keras.models import load_model\n",
    "# s = reset_tf_session()\n",
    "# last_finished_epoch = 10\n",
    "# model = load_model(model_filename.format(last_finished_epoch))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Training takes **2 hours**. You're aiming for ~0.93 validation accuracy."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2017-09-03T14:23:36.792701Z",
     "start_time": "2017-09-03T13:00:45.496194Z"
    },
    "collapsed": true,
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "# fine tune for 2 epochs (full passes through all training data)\n",
    "# we make 2*8 epochs, where epoch is 1/8 of our training data to see progress more often\n",
    "model.fit_generator(\n",
    "    train_generator(tr_files, tr_labels), \n",
    "    steps_per_epoch=len(tr_files) // BATCH_SIZE // 8,\n",
    "    epochs=2 * 8,\n",
    "    validation_data=train_generator(te_files, te_labels), \n",
    "    validation_steps=len(te_files) // BATCH_SIZE // 4,\n",
    "    callbacks=[keras_utils.TqdmProgressCallback(), \n",
    "               keras_utils.ModelSaveCallback(model_filename)],\n",
    "    verbose=0,\n",
    "    initial_epoch=last_finished_epoch or 0\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "## GRADED PART, DO NOT CHANGE!\n",
    "# Accuracy on validation set\n",
    "test_accuracy = model.evaluate_generator(\n",
    "    train_generator(te_files, te_labels), \n",
    "    len(te_files) // BATCH_SIZE // 2\n",
    ")[1]\n",
    "grader.set_answer(\"wuwwC\", test_accuracy)\n",
    "print(test_accuracy)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "# you can make submission with answers so far to check yourself at this stage\n",
    "grader.submit(COURSERA_EMAIL, COURSERA_TOKEN)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "That's it! Congratulations!\n",
    "\n",
    "What you've done:\n",
    "- prepared images for the model\n",
    "- implemented your own batch generator\n",
    "- fine-tuned the pre-trained model"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  },
  "toc": {
   "colors": {
    "hover_highlight": "#DAA520",
    "navigate_num": "#000000",
    "navigate_text": "#333333",
    "running_highlight": "#FF0000",
    "selected_highlight": "#FFD700",
    "sidebar_border": "#EEEEEE",
    "wrapper_background": "#FFFFFF"
   },
   "moveMenuLeft": true,
   "nav_menu": {
    "height": "120px",
    "width": "252px"
   },
   "navigate_menu": true,
   "number_sections": true,
   "sideBar": true,
   "threshold": 4,
   "toc_cell": false,
   "toc_section_display": "block",
   "toc_window_display": false,
   "widenNotebook": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
